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Abstract 

One of the most important factors for improving the personalization aspects of learning systems is to enable 
adaptive properties to them. The aim of the adaptive personalized learning system is to offer the most 
appropriate learning path and learning materials to learners by taking into account their profiles. In this paper, a 
new approach to adaptive personalized e-learning systems is proposed. This approach provides navigation on 
course materials, as well as navigations on course topics and domain concepts. In this approach, in addition to 
the difficulty of learning materials and course topics, students’ levels of knowledge and students’ understanding 
degrees on the course topics have also been taken into account. The Item Response Theory and Law of Total 
Probability have been used for estimating understanding degrees. The performance evaluation of the proposed 
approach has been tested by using the exam and project results, as well as grade point averages of the students 
from the Computer and Instructional Technology Department. The test results show the accuracy of the proposed 
method. It is believed that this study can improve the effectiveness of the adaptive e-learning system. 

Keywords: personalized learning, adaptive learning, e-learning, understanding degree 

1. Introduction 

One of the most important factors for improving the personalization aspects of learning systems is to enable 
adaptive properties to them. The aim of the adaptive personalized learning system is to offer the most 
appropriate learning path and learning materials to learners by taking into account theirs profiles. These systems 
enhance the usability of learning materials, and thus make the e-Learning system more effective. As a 
consequence, the learners’ acquisition of knowledge will be improved; this will lead to a more effective learning 
process (Esichaikul et ah, 2011). In the adaptive content aggregation method proposed in Modritscher et al. 
(2004), different types of content that depend on the learning and teaching style of a learner are offered; whereas 
adaptive presentation can be enriched with additional explanations, adaptive navigation can be adapted through 
global or local guidance and orientation. 

The paper Munoz and Oliveira (2004) describes the main steps taken in developing an Adaptive Web Training 
Environment, which consists of the LOM standard, Domain and Student Knowledge Models, and Web 
ontologies. The Web-based Intelligent Tutoring System, which allows students for a dynamic generation of 
suitable courseware and provides adaptive feedback, is presented in Kosba and Boyle, (2003) 

Methods and Models based on the Probabilistic theory are widely used in the development of tutoring systems. 
Detailed information on the related topic can be found in the Woolf (2009). A Personalized e-learning system 
using the Item Response Theory is proposed in Shih and Tseng (2009). To recommend learners appropriate 
course materials that are based on their individual requirements, the authors use the item characteristic function 
with difficulty parameters. The idea, presented in this paper is improved in Chen and Duh (2008). The Fuzzy 
item response theory, which is considered to be capable of helping to recommend courseware with suitable 
difficulty levels for learners according to learner’s fuzzy feedback responses, is proposed by the authors of this 
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study. Authors of the study Baylari and Montazer (2009) propose a personalized multi-agent e-learning system 
which can be used to estimate learner’s ability by means of the item response theory, and then to present a 
personalized and adaptive post-test based on that ability. Also the system can assist the discovery of learner’s 
learning problems via learner responses on review tests through using an artificial neural network (ANN) 
approach and then appropriate learning materials can be recommended to the student. 

The proposed framework in the Dag and Erkan (2010) is aimed to form a personalized learning environment. 
The designed learning system according to this framework includes the domain model, user model, and 
adaptation model. The components of the system have been realized by an ontology-based knowledge modelling 
approach. The problems related to the integration of learning standards, Semantic Web, and adaptive 
technologies to meet the learner requirements, are discussed in Jovanovic at al. (2009). 

As a result of a brief review on adaptive personalized systems, we can conclude that, in most systems, 
adaptability is achieved by providing appropriate course materials to learners in accordance with their profiles. 
Meanwhile, course materials are not the only factor influencing learning outcomes. One of the reasons that the 
learner does not understand the courseware may be that he/she does not have enough prerequisite knowledge 
about the topic to be learned. Unfortunately, one way of adapting this aspect is often neglected. 

The approach proposed in this paper takes into account the difficulty of learning materials and course topics, 
students’ knowledge levels as well as their understanding degrees of the course topics; and it also provides with 
a navigation of the course materials, as well as navigations of the course topics and domain concepts. The 
approach makes use of the Item Response Theory and The Law of Total Probability for estimate of 
understanding degrees. The offered approach has been carried out in the Adaptive Learning Management System 
developed by the authors. 

The paper has been structured as follows: The background knowledge on the research subject is given in Section 
2. In Section 3, the proposed approach is explained. Some experimental results related to the subject area are 
analysed in Section 4. Finally, the conclusion is given in Section 5. 

2. The Background Knowledge 

2.1 Item Response Theory 

The Item response theory (IRT) is a probabilistic model, which gives the probability that a person with a given 
ability level will answer correctly (Baker, 2001). The simplest form of IRT is the One-parameter Logistic Model, 
also known as the Rasch Model (Baker), which posits that the probability of a person with ability 0, answering 
an item with difficulty h correctly depends on both the ability of the person and difficulty of the item. In other 
words, if a person has a high ability in a particular field, he or she will probably get an easy item correct. 

The Rasch model is expressed by the following formula: 


Pi (6) = 


e D(0—bj) 
l +e ( 9-b i) 


( 1 ) 


where Pi(0) is the probability that a randomly chosen examinee with ability 9, answers the item i correctly; 
e is a mathematical constant, approximately equal to 2.71828, that is the base of the natural logarithm; 

0 is the ability of a person; 

bj is a difficulty parameter of the item i, and 

D is a scaling constant usually set equal to 1.702 

2.2 The Law of Total Probability 

In the probability theory, the law of total probability is a fundamental rule, relating marginal probabilities to 
conditional probabilities. The law of total probability is the proposition that if {5„_:n=l,2,3,...} is a finite 
partition of a sample space, and for each event B n _ is measurable, then the probability of any event A of the 
same probability space is: 

P(A)=2]„(P(A/5„) P(B n ) (2) 

where P(A/l? n ) is the conditional probability of event A oil B n ; 

P(B,J - The occurrence probability of event B„ (Wikipedia , 2012). 

2.3 Ontology’ 

Gruber (1993) defines ontology as “the specification of a conceptualization”. Therefore, ontology is a 
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description of concepts and their relationships. It permits the structuring of a knowledge domain. Ontologies are 
commonly used for describing the metadata of learning resources in order to help with the searching of learning 
objects in data repositories. Course ontology is used to describe the relations among course subjects. The 
ontology can be expressed as a graph G(T,R), where T is a set of the nodes which represent the subjects, and R is 
a set of semantic relationships between the pairs of nodes t and k (t,keT). Each edge can be assigned a weight. 
The fragment of the graphical depiction of the subject ontology on the course of Database Management Systems 
(DBMS) is given in Figure 1. 



Figure 1. Graphic representation of the concept ontology for the DBMS course 
Definitions of some concepts used in the suggested method 

Definition 1. Difficulty level of a learning material: DM 1 represents the difficulty level of the /th learning 
material. According to the LOM standard, the difficulty of a learning object is determined according to “how 
hard it is to work with or through this learning object for the typical intended target audience” (IEEE, 2004). To 
evaluate the difficulty level of a learning object, the Standard recommends 5 linguistic values: {very easy, easy, 
medium, difficult, very’ difficult}. Hereafter, we will use the abbreviations of these terms as {VE, E, M, D, VD}, 
respectively. 

Definition 2. Difficulty level of course concept: DC' represents the difficulty level of the /th course topic. We 
use this parameter to determine “how difficult it is to understand this course concept”. This parameter takes a 
value from the set of {very’ easy(VE), easy(E), medium(M), difficult(D), very’ difficuIt(VD)}. 

Definition 3. Student’s Knowledge Level: KWL ; shows the knowledge level of the /th student. This parameter 
represents the knowledge level of a student on the subject area that he/she wants to learn. The results obtained 
with different knowledge assessment methods, such as the students' grade point average (GPA), exam grades, 
and preliminary test results can be used to determine the level of knowledge. We will use five levels of 
knowledge, namely {very low (VL), low(L), medium(M), high(H), very’ high(VH)}. 

Definition 4. Understanding degree of course topic: This parameter determines how much and to what extent 
the topic was understood by the target student. 

The understanding degree of the student j of topic k-Uf can be expressed as 

U^=f{KWLj, DM 1 , DC k ) (3) 
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Where KWLj is the knowledge level of student j; 

DM represents the difficulty of learning material i as recommended for the study of topic k; 

DC 1 ' is the difficulty of the topic k. In the proposed method this parameter is associated with the key concept of 
the topic. . Key concepts are determined by domain experts. 

The formula (3) means that understanding degree of course topic depends on the following three factors: 
student’s knowledge, difficulty of the concepts referred in the topic, and difficulty of the appropriate learning 
material. 

The process of estimating of understanding degrees is described in Figure 2. 

We use the following five linguistic values to estimate understanding degree: {very’ low (VL), low (L), medium 
(M), high (H), very high (VH)}. 

After the necessary primary knowledge is given, we can describe the approach for Adaptive Personalized Course 
Learning System (APCL). 



Figure 2. Estimating of understanding degree 


3. The Approach for Adaptive Personalized Course Learning System 

This approach allows for choosing an effective way of learning with regard to parameters such as students’ 
knowledge level, and understanding degree, as well as the difficulty of course topics, and course learning 
materials. APCL is based on the use of ontological knowledge about course contents, and also on the application 
of the probabilistic methods, such as the Item Response Theory, and Law of Total Probability. 

A brief description of the proposed APCL is given below. 

The initial values of understanding degrees are determined for each knowledge level of each course topic. For 
this purpose, the IRT model is used. The value of understanding degree on the topic is recalculated by means of 
the Law of Total Probability, as based on the appropriate values of the previous topics, 

The implementation process of the approach consists of the following steps. 

Step 1. A Knowledge Base is created for the APCL. The implementation of the approach requires some 
information about learners’ profiles, courses to be learned, and learning materials on each course topic. The 
following steps should be taken to create the knowledge base that consists of this information: 

- the course outline, which reflects the hierarchical structure of the course topics to be learned, is 
determined; 

- for each topic, the key concepts are determined, and as a consequence the list of all concepts for the target 
course are formed; 

- a concept ontology, which covers all these concepts and reflects the relationship between them, is created; 

- for each course, a list of learning materials is created. Difficulty levels of these materials are determined 
and the learning materials are ordered by this parameter. 


95 


















www.ccsenet.org/ies 


International Education Studies 


Vol. 6, No. 5; 2013 


Step 2. By using the Item Response Theory (IRT) method, the understanding degrees of the course topics for 
each knowledge level are estimated. We make the assumption that the knowledge is used to measure a student's 
ability. Due to the fact that knowledge levels and difficulty levels are expressed in linguistic values, it is also 
needed to convert these values to the appropriate numeric values. For this purpose, the simple conversion tables 
(Table 1 and Table 2) are used. 


Table 1. Linguistic and numeric values for Knowledge Levels 


Linguistic 

VL 

L 

M 

H 

VH 

Numeric 

-2 

-1 

0 

1 

2 

Table 2. Linguistic and numeric values for Difficulty 

Levels 





Linguistic 

VE 

E 

M 

D 

VD 

Numeric 

-2 

-1 

0 

1 

2 


Step 3. A student logins the System and enters the title of the topic that he wants to learn. The values of 
student’s knowledge level and the difficulty of the topic are retrieved from the Knowledge Base according to the 
student’s identity information, and to the topic. On the basis of these data, the value of the appropriate 
understanding degree is determined. 

Step 4. If the understanding degree of the student is low, the System offers to the student the learning material 
with lower difficulty, if not, the learning process is continued on the original list of the learning materials. 

Step 5. The student begins to learn the course topic from the recommended learning material. 

Step 6. The knowledge of the student on the topic is tested. On the basis of the test results, the student's 
knowledge level is re-evaluated. 

Step 7. If the student wants to learn the next topic, the system re-calculates the understanding degree on the topic, 
taking into account the new value of the knowledge level. This step is carried out by the LTP module 

Step 8. If the understanding degree is low, then the system recommends the student to improve his/her 
knowledge on the predecessor concepts. The list of the concepts is determined through the concept ontology. 
The student may decide to go to the study of the next topic or to improve their prerequisite knowledge. 

Assume that the student learns a topic, for which the keyword is “3NF”. The understanding degree of the 
student with knowledge level ‘‘medium” is equal to 0.3. As the understanding degree is low, the system 
encourages the student to improve their knowledge by learning the predecessor concepts INF, 2NF, Functional 
Dependency, Transitive Dependency, and Key Attributes. 

Step 9. The student starts studying the new topic. 

Step 10. The aim of this step is to determine whether the student has understood the reading material or not. This 
aim is achieved through an interactive query. The System helps the learner to answer questions such as “ what 
should I read to learn the topic”, ‘‘what should I do if the proposed material is difficult to understand or 
materials offered to me are very ; simple ”. 

When the learner completes the reading of the offered material, the System asks the student whether he/she has 
understood the content of the material or not. Considering that the answer is positive, the system offers him/her 
some test materials to measure his/her knowledge level on the related topic. By taking into account the new 
value of the knowledge level, the understanding degree is re-calculated through the Law of Total Probability. 
Taking into consideration the new value of the understanding degree, the System gives advice to the students on 
proceeding with the next topic or not. If the value of understanding degree is low, the system makes suggestions 
about topics as well as to what extent the student should increase his/her knowledge level. This action can be 
interpreted as that while increasing knowledge on the previous related topics, the probability of understanding a 
topic increases. 

If the answer of the student is negative, the system tries to identify why student cannot understand the learning 
material. For this purpose, the system offers the student the following four options: 
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a) The material is very theoretical; it could have been clearer with practical examples; 

b) I could not understand the following terms (the terms are marked or entered by the learner); 

c) I could not understand a particular part of the material (the related part of the learning material is 
marked); 

d) I didn ’t understand anything 

Case (a): In this case the system offers a new learning material from the ordered list of the learning materials. 

Case (b): The system selects the learning objects, from among definitions of the marked terms, and the student is 
advised to read these objects. 

Case (c): The system analyses the content of the selected part, and the student is offered a learning object with a 
similar content. 

Case (d): In this case, the system will recommend the student to learn the previous topics. 

Assume that the learner did not understand the concept “ second normal form ”, and he/she chose the option (b). 
In this case the system, moving on the course ontology, determines that the concepts ‘ functional 
dependency" and “first normal form’’'' are the parents of the "second normal form”. Therefore, the learning 
materials that describe the parent concepts are recommended for learning (Figure 3). 

Below we describe the procedures for the implementation of the above-described actions. 


Nomializatia: 



Functional Dependency First Normal Form 

Figure 3. An example of Flierarchic relations between the DBMS concepts 


Procedure Understanding Degree ( student, 
input: Student, Course 
output: understanding degree 

Symbol Definition: 

Student 

Course 

Knowledge level 
Topic 

Difficulty level 
Understanding degree 
Predconcepts 
Keyconcept 

DataBase Tables: 


course, understanding degree) 


/* the identification number of the student 
/* the identification number of the course, to be learned 
/* the knowledge level of the student 
/* the identification number of the topic, to be learned 
/* the difficulty level of the topic 
/* the understanding degree of the student 
/* the set of predecessor concepts of the concept 
/* key concept of the topic 
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Student_knowIedge_table /* contains students profile knowledge 

Understanding_degree_table /* contains understanding degrees 

Course Jable: /* contains course knowledge 

Procedures: 

IRT /* calculates the initial value of the understanding degree, depending on learners’ knowledge level as 
well as the difficulty level of the topic to be learned 

LTP /* recalculates the understanding degree of the topic by means of the Law of Total Probability, in 
accordance with the initial understanding degrees of the parent topics, and closeness between the topics and their 
parents 

Actions: 

Retrieve knowledge_level from the learner_profileJable for the student 

Convert linguistic value of the knowledge level to the numeric value by using the Table 1 

Retrieve difficulty Jevel from the course_topic_table for the topic 

Convert linguistic value of the difficulty level to the numeric value by using the Table 2 

Call IRT (knowledgeJevel, difficulty Jevel, understanding_degree) 

INSERT understanding degree of the topic for the student INTO learner_course Jable 
Retrieve keyconcept of the topic from the topic concept Jable 
Retrieve understanding_degree from the concept Jable for the keyconcept 
u_degree_concept= understanding degree 

Retrieve predconcepts of the concept from the concept ontology 

Retrieve understanding_degree for the all predconcepts of the keyconcept from the conceptjable 
u degree_predconcept = understanding degree 

Retrieve the value of closeness between the concept and the predconcepts from the concept ontology 

Do I 1 To_N pre( | concC pt 

{ 

Call LTV(u_degree_predconcept [I], closeness [I], u_degree[I]) 
u_degree_new= u_degree_new+ u_degree[I] 

} 

INSERT u_degree-new INTO the learner_course Jable 

End of Procedure 

Procedure IRT (T,B,P) /* calculate the understanding degree 

Symbol Definition 

T /* ability level; the parameter receives a value from the set of {-2, -1, 0, 1, 2} 

B /* item difficulty; the parameter receives a value from the set of {-2, -1,0, 1,2} 

D /* scaling constant (0=1.702) 

Calculate 

P=EXP (D*(T-B))/(1+EXP(T -Bj) /*P means the probability that students with ability T can understand the item 
with difficulty level B 

End of the Procedure 

Procedure LTP(PA,PB,PAB) /calculates the understanding degree of the student on the given concept through 
the Law of Total Probability. 

Symbol Definition 

PA /* the probability of the event A 
PB /* the probability of the event B 
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PAB /* the probability of the event A given B 

Calculate 

PA=PAB*PB 

End of the procedure 

4. The Experimental Results 

Performance evaluation of the proposed method was carried out on the course of the Database Management 
System, taught in the Computer Education Department. The following procedure was performed for this 
purpose. 

-First, a detailed list of course topics has been created. 

-Then, the keyword concepts and their predecessor concepts have been determined for each topic (Table 3). 

-The Concepts Ontology, which includes relationships among about 40 database concepts, has been created. 

-The learning materials of the DBMS were selected and saved in the Learning Object Repository. 

Grade Point Averages (GPA) of the students were used for estimating the students’ knowledge levels. 


Table 3. Key concepts and theirs predecessors 


Concepts 



Predecessor Concepts 


Normalization 

INF 

2NF 

3NF 

relation 

attribute 

INF 

UNF 

relation 

Repeated group 

attribute 

Key attribute 

2NF 

INF 

Functional 

dependency 

Complex key 

Partial functional 
dependency 

Full functional 
dependency 

3NF 

INF 

2NF 

Functional 

dependency 

Transitive functional 
dependency 

Key attribute 

Data Model 

Model 

Data 

relationship 

Data expression 

abstraction 

Relational 

Model 

Data Model 

Relation 

attribute 

key 

Set operations 

ER diagram 

entity 

relationship 

ER notation 

attributes 

Conceptual 

model 

EER diagram 

ER diagram 

EER notation 

generalization 

specialization 

inheritance 

Simple SQL 
statement 

Table 

SELECT 

FROM 

WHERE 

SQL structure 

Relational 

Algebra 

relation 

RA operations 

RA notation 

attributes 

tuple 

RA Basic 
operations 

Projection 

Selection 

Cartesian 

Product 

Intersection 

Union 


The test data consisted of more than 200 students’ grades from exams, midterm exams’, and semester projects. 
All of the students were studying at the Computer and Instructional Technology Department, as the study was 
conducted. On the basis of these data, and observations of the authors, difficulty levels were determined for all of 
the course topics (Table 4). 
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Table 4. Difficulty levels and students average grade points on the topics 


Course topics 

Difficulty 


Knowledge level 



Level 

low 

medium 

high 

Very high 

Data Models 

E 

0.32 

0.82 

1 

0.90 

Entity-relationship model 

M 

0.41 

0.63 

0.81 

0.92 

Extended Entity-relationship model 

D 

0.0 

0.43 

0.71 

0.80 

Relational Algebra 

D 

0.0 

0.41 

0.70 

0.91 

Relational Model 

M 

0.08 

0.84 

0.85 

0.88 

INF 

M 

0.14 

0.62 

0.92 

1 

2NF 

VD 

0.0 

0.21 

0.71 

0.71 

3NF 

VD 

0.0 

0.31 

0.53 

0.73 

SQL (simple queries) 

E 

0.34 

0.82 

0.93 

1 

SQL (complex queries) 

D 

0.22 

0.50 

0.80 

0.90 

Query creating in Access DBMS 

VE 

0.45 

0.94 

1 

1 


The participants’ GPAs and understanding degrees as obtained by means of the proposed APCL method are 
given in Table 5. Corresponding graphic representations are given in Figure 5. As can be seen in these table and 
graphics, the exam results, and the results obtained through the APCL method are very close to each other. This 
indicates the validity of the proposed method. 


Table 5. Comparison average grade points of students and the their understanding degrees obtained by the APCL 
method 


Difficulty levels of the topics 


Very Easy 


Easy 

Medium 

Difficult 

Very Difficult 

Students’ 

knowledge 

levels 

GPA 

Underst. 

degree 

GPA 

Underst. 

degree 

GPA 

Underst. 

degree 

GPA 

Underst. 

degree 

GPA 

Unders. 

degree 

VL 

0.05 

0.50 

0.01 

0.27 

0.00 

0.12 

0.00 

0.05 

0.00 

0.01 

L 

0.45 

0.73 

0.33 

0.50 

0.11 

0.27 

0.00 

0.12 

0.00 

0.05 

M 

0.94 

0.88 

0.82 

0.73 

0.69 

0.50 

0.44 

0.27 

0.25 

0.12 

H 

0.99 

0.95 

0.93 

0.88 

0.86 

0.73 

0.73 

0.50 

0.62 

0.27 

VH 

0.99 

0.98 

0.99 

0.95 

0.90 

0.88 

0.86 

0.73 

0.71 

0.50 
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Very Difficult 


0,8 



e) Difficulty level is “very difficult” 

Figure 5. Comparison of average grades and understanding degrees 


The proposed approach has been realised in the Adaptive Personalized Leaning Management System,. The 
System is realised through Moodle, which is a free software platform for developing Learning Management 
Systems. Moodle is a software package for producing Internet-based courses and web sites (Moodle, 2012). To 
realize the proposed approach, functionality of this program has been extended by adding new modules. One 
screenshot of the APCL application at run-time is given in Figure 6. 
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‘ Adaptive Learning Sy X _ ” 01 

C ri Q botenesne.comu.edu.tr/mod/lti/view.php7id=4 H t 

» D Digeryi 


Muzaffer OZDEMtP. olarak girij yaptiniz (Ciki;) 

Adaptive Learning System 



Adaptive Personalized Learning System 

[Course :Database Management Systems 


opic:Third Normal Form 


® I could not understand the following terms 
[repeating elements 

O I could not understand a particular part of the 
material 

O I do not understand anything 


his is meant to be a brief tutorial aimed at beginners who want to get a conceptual grasp on die database 
normalization process. I find it difficult to \isualize these concepts using words alone, so I shall rely as much 
as possible upon pictures and diagrams. 

To demonstrate die main principles involved, we will take die classic example of an Invoice and level it to 

the Third Normal Form. 

Important Note: This is not a description of how you would actually design and implement a database. The 
sample database screenshots are not meant to be taken literally, but merely as visual aids to show how the 
raw data gets shuffled about as the table structure becomes increasingly normalized. 


Purists and academics may not be interested in this treatment. I will not cover issues such as the benefits and 
drawbacks of normalization. For those who wish to pursue the matter in greater depth, a list of references 
for further reading is presided at tile end. 

a For die most part, the first three normal forms are common sense. When people sit down to design a 

database, they often already have a partially-normalized structure in mind — normalization is a natural way 
of perceiving relationships between data and no special skill in m at h ematics or set theory is required. 

In fact, whereas normalization itself is intuitive, it usually takes quite a bit of advanced skill to recognize when 
it is appropriate to r/e-normalize a database (that is, remove the natural efficient relationships that a 
normalized data structure prosides). Denormalization is a fairly common task, but it is beyond the scope of 


Figure 6. One Sceenshot of the APCL 


5. Conclusion 

In this study an approach was proposed for adaptive personalized course learning system - APCL. The approach 
is based on the probabilistic models such as the Item Response Theory and The Law of Total Probability, and 
offers a way of improving learning by increasing the understanding degree of topics to be learned. The main 
feature of this approach is that adaptively of the learning system is achieved by navigating through not only 
learning materials, but also the course concepts. For this purpose, a new concept-understanding degree of course 
topics has been introduced, the value of which depends on the student’s knowledge, difficulty of the concepts 
referred in the topic, as well as the difficulty of the appropriate learning material. The implementation process of 
the APCL has been described. Furthermore, a description of the system through which the proposed approach 
has been realised is given. The System has been developed on the Learning Management System Moodle. The 
program modules that perform adaption functions have been written and added to the Moodle. Performance 
evaluation of the proposed method was carried out during the course of Database Management System taught by 
authors in the Computer and Instructional Technology Department. A positive correlation of the relationship 
between understanding degrees and knowledge levels, and a negative correlation of the relationship between 
understanding degrees and difficulty levels were confirmed by both methods. Therefore, it would not be wrong 
to claim that the proposed approach can be successfully applied in the development of adaptive learning systems. 
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